Skip to content
This repository was archived by the owner on Oct 12, 2022. It is now read-only.
/ druntime Public archive

Conversation

@wilzbach
Copy link
Contributor

See also:

I didn't add any tests for Class deallocators
as they have been deprecated since D2 and we call the existing functions.

CC @JinShil

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.


$(CONSOLE
sed "s/delete \(.*\);/__delete(\1);/" -i **/*.d
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest adding something to the effect that this is intended to only be used as a last resort when destroy is not an option. At least that's my understanding: We want to migrate users to destroy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential verbage: "Users should prefer object.destroy to explicitly finalize objects, and only resort to __delete when object.destroy would not be a feasible option."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sed command could be a bit refined: s/\Wdelete[ \t]+\(.*\);/__delete(\1);/g

This function has been added to provide an easy transition from `delete`.
It performs the same functionality as `delete`.
$(REF destroy, object) is `@safe` and should be used if possible.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I think this should have some information explaining that users should first look to destroy and only to __delete when destroy is not an option.

src/foo.d Outdated
@@ -0,0 +1,151 @@
import std.stdio;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this file (src/foo.d) doing in this commit?

@ibuclaw
Copy link
Member

ibuclaw commented Jan 12, 2018

Looks like you committed an extra file by mistake

@ibuclaw
Copy link
Member

ibuclaw commented Jan 12, 2018

What advantages does this have over the (poorly named) destroy?

@JinShil
Copy link
Contributor

JinShil commented Jan 12, 2018

What advantages does this have over the (poorly named) destroy?

My understanding is it was mandated by @andralex. See discussion starting here

@ibuclaw
Copy link
Member

ibuclaw commented Jan 12, 2018

Still, my understanding was that (years ago when delete was first put for deprecation) destroy was added as a migration path - by Andrei himself I believe.

Obviously that function has failed spectacularly if this is needed. 🙄

@wilzbach
Copy link
Contributor Author

Still, my understanding was that (years ago when delete was first put for deprecation) destroy was added as a migration path - by Andrei himself I believe.
Obviously that function has failed spectacularly if this is needed. 🙄

A bit of history I could dug up, but it seems that destroy has been there for a long time:

#231
#7

memory.
If the garbage collector was not used to allocate the memory for
the instance, undefined behavior will result.
)
Copy link
Contributor

@JinShil JinShil Jan 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation just sets the pointer to null. I don't think that initiates an immediate call to the GC to free the memory.

a.test = "exists";
auto b = &a;
auto c = &b;
delete c;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be __delete?

auto b = a;
__delete(b);
assert(b is null);
assert(a == [1, 2, 3]);
Copy link
Contributor

@JinShil JinShil Jan 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm understanding correctly, you are accessing through a after b was __deleteed. According to the documentation comments, that's undefined behavior.

}
else
{
static assert(0, "It is not possible to delete: " ~ T.stringof);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"It is not possible to delete`" ~ T.stringof ~ "`"

@ibuclaw
Copy link
Member

ibuclaw commented Jan 12, 2018

A bit of history I could dug up, but it seems that destroy has been there for a long time:

231 is titled "Rename clear to destroy". So it could always be my memory that is incorrect. :-)

@jmdavis
Copy link
Member

jmdavis commented Jan 13, 2018

clear was talked about in TDPL. I don't know how old it is, but I don't think that it was new then. Regardless, it's never been about freeing memory, only about destroying the object and putting it in an invalid state, so it's really not the same as what this is trying to do - though in principle, the idea behind clear/destroy and getting rid of delete is that no one should ever do what delete does if they're using GC-allocated memory. Andrei has been adamant about that being a terrible idea for years now, but we've just never gotten rid of delete (if nothing else, because without allocators, trying to replace code that uses with code that does something similar with non-GC-allocated memory it is a royal pain).

I am a bit surprised thought that Andrei wants this delete-replacement. From what I recall, his stance in the past has always been that delete is fundamentally a bad idea for GC-allocated memory and that it should be gone, and keeping a library function around that does the same thing doesn't fit with that - though the fact that it's not built-in does reduce its visibility and make it seem less acceptable / more dangerous, so I guess that he considers that enough now.

Personally, I think that anyone looking to do something like this should not be using GC-allocated memory, but it's not like we're completely against letting folks risk blowing their own feet off - we just don't want it to be easy to do that by default.

`core.memory.__delete` has been added

$(REF __delete, core, memory) allows easy migration from the deprecated `delete`.
`__delete` behaves exactly like `delete` and will call the class dtor immediately
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete/__delete apply to structs and arrays as well, so please update the documentation


$(CONSOLE
sed "s/delete \(.*\);/__delete(\1);/" -i **/*.d
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sed command could be a bit refined: s/\Wdelete[ \t]+\(.*\);/__delete(\1);/g

}
else static if (is(T == struct))
{
_d_delstruct(cast(void**) &t, typeid(T));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this is incorrect. __delete should be only applicable to pointers to struct. Let's not innovate here.

}
else static if (is(T == U*, U))
{
t = null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should free memory

}
else static if (is(T : E[], E))
{
t = null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should free memory

@andralex
Copy link
Member

Still, my understanding was that (years ago when delete was first put for deprecation) destroy was added as a migration path - by Andrei himself I believe.

Obviously that function has failed spectacularly if this is needed. 🙄

Thanks for the quip. The olden idea is to keep the safer clear while still allowing people to free objects. That was (and is) possible by means of two function calls. More recently our doctrine for deprecation has become to provide a simple replacement for people who have delete spread out through their code and just want to keep the current semantics for the time being. So both clear and __delete have their place.

@wilzbach wilzbach force-pushed the __delete branch 4 times, most recently from 915aeab to 444ab75 Compare January 13, 2018 09:37
@wilzbach
Copy link
Contributor Author

Alrighty I went over the implementation again and it now matches with DMD 1:1 -> https://github.com/dlang/dmd/blob/v2.078.0/src/dmd/e2ir.d#L3886

The only thing that I couldn't figure out is how to detect at compile-time whether -profile=gc has been set by the user (DMD uses a different function mapping for this option - see toTraceGC), but I doubt that it's essential for this PR as -profile=gdc is mostly about detecting allocations.

`core.memory.__delete` has been added

$(REF __delete, core, memory) allows easy migration from the deprecated `delete`.
`__delete` behaves exactly like $(LINK2 $(ROOT_DIR)spec/expression.html#delete_expressions, `delete`):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the procedures we need to perform to deprecate and eventually remove delete is to remove all references to it in the spec (see dlang/dlang.org#1941). I suggest removing the link to the spec.

there is a destructor for that class, the destructor
is called for that object instance.
)
$(LI Otherwise, the garbage collector is called to immediately free the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "otherwise" is ambiguous because it's applied to a conjunction. Does it refer to the fact that t is a class object reference, to the fact there is a destructor, or to the conjunction? (It applies to the second clause only). I'll make an edit to clarify.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: That was taken 1:1 from the spec ;-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wilzbach @WalterBright and I are at POPL 2018 and one thing that became abundantly clear is we must overhaul the spec making it radically more precise. More on that later.

@andralex
Copy link
Member

@wilzbach I just sent a patch to the doc - we should reuse destroy both in the documentation and the implementation. Also it seems associative arrays have been forgotten.

$(UL
$(LI
Calls `.object.destroy(*x)` (if `x` is a pointer) or `.object.destroy(x)`
(otherwise) to destroy the referred entity.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's strictly speaking not true for class/interface objects and raw memory:

Exact behavior

  • If x is a class or interface object -> rt_finalize

druntime/src/rt/lifetime.d

Lines 163 to 167 in 098a08a

else
{
rt_finalize(cast(void*) *p);
}
GC.free(cast(void*) *p);

Of course, object.destroy does call rt_finalize:

druntime/src/object.d

Lines 2887 to 2896 in 098a08a

void destroy(T)(T obj) if (is(T == class))
{
rt_finalize(cast(void*)obj);
}
/// ditto
void destroy(T)(T obj) if (is(T == interface))
{
destroy(cast(Object)obj);
}

FTR rt_finalize is deprecated, but rt_finalize2 does call a few interesting things:

druntime/src/rt/lifetime.d

Lines 1382 to 1419 in 098a08a

debug(PRINTF) printf("rt_finalize2(p = %p)\n", p);
auto ppv = cast(void**) p;
if(!p || !*ppv)
return;
auto pc = cast(ClassInfo*) *ppv;
try
{
if (det || collectHandler is null || collectHandler(cast(Object) p))
{
auto c = *pc;
do
{
if (c.destructor)
(cast(fp_t) c.destructor)(cast(Object) p); // call destructor
}
while ((c = c.base) !is null);
}
if (ppv[1]) // if monitor is not null
_d_monitordelete(cast(Object) p, det);
if (resetMemory)
{
auto w = (*pc).initializer;
p[0 .. w.length] = w[];
}
}
catch (Exception e)
{
import core.exception : onFinalizeError;
onFinalizeError(*pc, e);
}
finally
{
*ppv = null; // zero vptr even if `resetMemory` is false
}

  • If x is a reference to a struct -> Typeinfo_Struct.destroy:

druntime/src/rt/lifetime.d

Lines 177 to 183 in 098a08a

extern (C) void _d_delstruct(void** p, TypeInfo_Struct inf)
{
if (*p)
{
debug(PRINTF) printf("_d_delstruct(%p, %p)\n", *p, cast(void*)inf);
inf.destroy(*p);

Here it's all about calling the dtor:

druntime/src/object.d

Lines 1194 to 1199 in 098a08a

if (xdtor)
{
if (m_flags & StructFlags.isDynamicType)
(*xdtorti)(p, this);
else
(*xdtor)(p);

  • If x is a reference to an arrays -> for all structs within: Typeinfo_Struct.destroy:

finalize_array(start, length, ti);

druntime/src/rt/lifetime.d

Lines 1347 to 1356 in 098a08a

void finalize_array(void* p, size_t size, const TypeInfo_Struct si)
{
// Due to the fact that the delete operator calls destructors
// for arrays from the last element to the first, we maintain
// compatibility here by doing the same.
auto tsize = si.tsize;
for (auto curP = p + size - tsize; curP >= p; curP -= tsize)
{
// call destructor
si.destroy(curP);

  • Otherwise only GC.free is called.

Frees the memory allocated for `x`. If `x` is a reference to a class
or interface, the memory allocated for the underlying instance is freed. If `x` is
a pointer, the memory allocated for the pointed-to object is freed. If `x` is a
built-in array or associative array, the memory allocated for the array is freed.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Associative arrays aren't allowed: https://run.dlang.io/is/CtfP8f

)
Params:
t = aggregate object that should be destroyed
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to myself: I will rename this to x then.

@schveiguy
Copy link
Member

schveiguy commented Jan 13, 2018

From my memory:

  1. clear/GC.free was intended as the replacement for delete. The problem with delete is that it wasn't possible to destroy things deterministically without also creating a dangling pointer. The required conflation of memory management with finalization/destruction was a faulty mechanism, especially in a GC-by-default language. I think the intention was to allow a library function, and stop reserving a keyword for such a thing.
  2. clear was renamed to destroy because .clear is typically the name people use for removing all elements of a container. Given how prevalent UFCS is in D, then this creates massive confusion.

IMO, the destroy/GC.free mechanism is too raw, however. Sometimes it's exactly what you want, and GC.free requires a pointer, destroy requires a value. You are using clunky mechanisms everywhere if you use this thing.

@wilzbach
Copy link
Contributor Author

@wilzbach I just sent a patch to the doc

Nice (I just removed your trailing white-space in case you were wondering why I force-pushed your commit).

@andralex
Copy link
Member

@wilzbach yah I wonder if we could submit a github issue - would be great if their editor supported trailing space elimination

@andralex
Copy link
Member

andralex commented Jan 13, 2018

@wilzbach agreed about the discrepancies - in this case we must work backwards: __delete IS (by fiat) "destroy then deallocate". To the extent it isn't we need to adjust either destroy or __delete.

@wilzbach
Copy link
Contributor Author

yah I wonder if we could submit a github issue - would be great if their editor supported trailing space elimination

https://github.com/isaacs/github

@wilzbach agreed about the discrepancies - in this case we must work backwards: __delete IS (by fiat) "destroy then deallocate". To the extent it isn't we need to adjust either destroy or __delete.

What do you suggest? Do you want to change __delete do call destroy?
_d_delclass still checks for deallocators as class deallocators haven't been deprecated yet.

I would simply change the wording to sth. like:

Calls .object.destroy(x) (if x is a class or interface object) or typeid(*x).destroy(x) (if x is pointer to a struct) to destroy the referred entity. Arrays of structs call typeid(*x).destroy(x) for each element.

@andralex
Copy link
Member

Yes, __delete must call destroy. The wording you propose is good, thanks.

$(LI
Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`typeid(*x).destroy(x)` (if `x` is pointer to a struct) to destroy the referred entity.
Arrays of structs call `typeid(*x).destroy(x)` for each element.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also state: Arrays of structs call typeid(x).destroy(&x) for each element (that's what the implementation does below).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be .object.destroy(e) for each element e in the array.

assert(a is null);
}

// __delete returns with no effect if x is null
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a simple way to add a test this for "no effect"?

$(UL
$(LI
Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`typeid(*x).destroy(x)` (if `x` is pointer to a struct) to destroy the referred entity.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it should just be .object.destroy(*x), no? It's faster and better documented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about object.destroy being faster?

TypeInfo_Struct.destroy:

druntime/src/object.d

Lines 1192 to 1201 in 18be8bf

final override void destroy(void* p) const
{
if (xdtor)
{
if (m_flags & StructFlags.isDynamicType)
(*xdtorti)(p, this);
else
(*xdtor)(p);
}
}

object.destroy:

druntime/src/object.d

Lines 2954 to 2965 in 18be8bf

void destroy(T)(ref T obj) if (is(T == struct))
{
_destructRecurse(obj);
() @trusted {
auto buf = (cast(ubyte*) &obj)[0 .. T.sizeof];
auto init = cast(ubyte[])typeid(T).initializer();
if (init.ptr is null) // null ptr means initialize to 0s
buf[] = 0;
else
buf[] = init[];
} ();
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The former is an indirect call and the second is a direct call. Truth be told, destroy does more work because it reinitializes the object. In the case of __delete we don't care to reinitialize the object because we just wrote in the documentation that access after __delete is undefined behavior.

So all in all the best way to go is to write in the documentation that we call destroy and actually call _destructRecurse. That covers all defined cases and is fast.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_destructRecurse is private and in object - so we can't even do package(core).

$(LI
Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`typeid(*x).destroy(x)` (if `x` is pointer to a struct) to destroy the referred entity.
Arrays of structs call `typeid(*x).destroy(x)` for each element.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be .object.destroy(e) for each element e in the array.

// See also: https://github.com/dlang/dmd/blob/v2.078.0/src/dmd/e2ir.d#L3886
static if (is(T == interface))
{
(cast(Object) x).destroy;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should do what the spec says it does: .object.destroy(cast(Object) x); This is a pedantic point because Object does not define a method destroy.

}
else static if (is(T == class))
{
(cast(Object) x).destroy;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, .object.destroy(cast(Object) x); would clarify lookup

else static if (is(T == U*, U))
{
static if (is(U == struct))
typeid(U).destroy(x);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned, I think this is less efficient - it entails an indirect call instead of simply calling the destructor (if defined) or doing nothing (if not). I suggest: .object.destroy(*x)

static if (is(E == struct))
{
foreach (ref e; x)
typeid(E).destroy(&e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here


static if (is(T == interface) || is(T == class) || is(T == U2*, U2) || is(T : E2[], E2))
{
GC.free(&x);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this in the case of classes and interfaces. For those shouldn't the call be GC.free(cast(void*) x)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tests for GC.free

$(UL
$(LI
Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`(*x).__xdtor()` (if `x` is pointer to a struct and a custom destructor exists) to destroy the referred entity.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's exactly what we do, but I wasn't whether there's a more elegant way to put it - __xdtor is a bit ugly, but it's still better than ".object.destroy(*x), but doesn't initialize the struct to T.init"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People cannot be expected to know what __xdtor does, but they do know what a destructor is. So the text should be:

"Calls the destructor ~this() for the object referred to by x (if x is a class or interface reference) or for the object pointed to by x (if x is a pointer to a struct). If no destructor is defined, this step has no effect."

@wilzbach
Copy link
Contributor Author

So are we ready to 🚀 here?

Copy link
Member

@andralex andralex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more nits to look at.

pragma(mangle, "free") void fakePureFree(void* ptr);
}

extern(C) private @system nothrow
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are @nogc as well

The `delete` keyword allowed to free GC-allocated memory.
As this is inherently not `@safe`, it has been deprecated.
This function has been added to provide an easy transition from `delete`.
It performs the same functionality as `delete`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of assuming people know delete and defining __delete in terms of it, describe __delete as "destroy then deallocate" and at the very end "see also the deprecated keyword" etc.

$(UL
$(LI
Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`(*x).__xdtor()` (if `x` is pointer to a struct and a custom destructor exists) to destroy the referred entity.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People cannot be expected to know what __xdtor does, but they do know what a destructor is. So the text should be:

"Calls the destructor ~this() for the object referred to by x (if x is a class or interface reference) or for the object pointed to by x (if x is a pointer to a struct). If no destructor is defined, this step has no effect."

Calls `.object.destroy(x)` (if `x` is a class or interface object) or
`(*x).__xdtor()` (if `x` is pointer to a struct and a custom destructor exists) to destroy the referred entity.
Arrays of structs with a custom destructor call `x.__xdtor()` for each element in the array.
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Arrays of structs call the destructor, if defined, for each element in the array."

void __delete(T)(ref T x) @system
{
static void _destructRecurse(S)(ref S s)
if (is(S == struct))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unindent

{
static if (__traits(hasMember, S, "__xdtor") &&
// Bugzilla 14746: Check that it's the exact member of S.
__traits(isSame, S, __traits(parent, s.__xdtor)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the bugzilla is fixed, why do we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because fixing 14746, added this to _destructRecurse: 1a2290b

Copy link
Member

@andralex andralex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @wilzbach

@wilzbach wilzbach force-pushed the __delete branch 2 times, most recently from a675d5c to 57cbf25 Compare February 6, 2018 23:13
@wilzbach
Copy link
Contributor Author

wilzbach commented Feb 6, 2018

ping @wilzbach

Sorry. Pong. Addressed the final nits & squashed everything into one commit.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants